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ABSTRACT 

This paper presents a framework for the evaluation of concept maps. We focus on supporting the student in dealing with 
ill-structured and complex problems. We argue that these problems require the application of the modularity approach. In 
view of this, the paper describes ways of providing student support for the implementation of this approach within the 
context of a concept mapping activity. As a basis we make use findings from the field of brain science and mathematical 
concepts and tools from the field of network theory, in order to present specific evaluation methods, the further 
development of which we consider extremely promising. 
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1. INTRODUCTION 

In recent years we’ve increasingly witnessed a transition from the storage model toward the development of 
high-level skills (Perkins, 2000) in thinking and creativity. The ideas and the theoretical background 
underlying this transition are not new. They were first shaped at the beginning of the previous century 
(Dewey, 1925) and have been further confirmed and specialized by younger researchers in the field of 
learning. Cognitive tools, such as concept maps, play an important role in these approaches releasing the 
student from dealing with low-level mental functions and encourage a more creative way of thinking and 
synthesizing knowledge. 

This article aims to contribute to this field by presenting a framework and proposing guidelines for the 
assessment of concept maps. Research from three different fields is made use of in order to create the 
framework: a) the field of learning, b) the field of brain science and c) the field of network theory. The article 
attempts to answer the following questions: 

Why and how do students learn through the construction of concept maps? 

In what ways could we support students and what kind of feedback and suggestions could we provide 
them with? 


2. LEARNING AS STRUCTURE CREATION 

We can educate students to represent and organize the information they receive more efficiently. This goal 
could be achieved through either strong or weak methods. Weak methods refer to methods which are 
independent of content. Particularly when solving problems that deal with the capacity for innovation and 
navigation within ill-structured domains, the use of such methods is essential. Strong methods refer to 
methods which are dependent on the knowledge domain. Specific formulas, algorithms, classifications, etc., 
help us to very quickly reach the solution of a problem on the condition, of course, that the particular 
problem is well structured. 

Experts differ from beginners in the strong and weak methods they possess and apply. Experts organize 
information more efficiently and as a result the solution space becomes more coherent. In this context, 
evaluation methods that allow us to observe the process of the creation of structure by the student acquire 
particular importance. This means observing possible weak and strong methods which the student applied in 
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order to solve the problem. In the case of concept mapping: a) strong methods are reflected in the schema 
(diagram) produced by the student and are expressed mainly on a semantic level, while b) weak methods 
could be considered to be also portrayable in the characteristics of the structure produced. 

A structure may or may not be modular. A non-modular structure of the natural world cannot be 
characterized as “well- structured” unless it is very simple (trivial) or if not assessed in relation to dynamic 
growth and enrichment. However, learning is an entity not subject to closure. Learning constitutes the 
creation of structures, the main qualities of which are the potential for expansion and enrichment. Its main 
feature is the change produced when the framework expands. 

Structures with such features have documented in various ways (mental constructions, simulations, 
visualizations of complex structures, etc.) that the modular way of development is more efficient and 
effective (Newman, 2010; Lipson, Pollack & Suh, 2002) for a variety of reasons such as easy error recovery, 
easier expansion, etc. The human brain is modular both structurally as well as functionally (Bullmore & 
Sporns, 2009). This feature allows it to evolve and to enrich its structure. There are indications (Bassett, 
Wymbs, Porter, Mucha, Carlson, & Grafton, 2011; Fair, Cohen, Power, Dosenbach, Church, Miezin & 
Petersen, 2009) that when the brain undergoes learning processes, it becomes more modular. 


3. SEARCHING FOR STRUCTURE IN A CONCEPT MAP 


3.1 Finding Modules 

For the detection of modules, namely the ideas contained in a map, we could build on the progress that has 
been made over the last decade in the area of network theory, both on a theoretical as well as on an 
application level. The obvious premise for finding modules is that the concepts that compose the ideas 
contained in a map are connected to each other in a different way than with the meanings that are outside the 
framework that the idea defines. To be more specific, we can make the self-evident assumption that there 
should be a high density of connections among concepts within the idea, in relation to concepts that are 
outside it. Based on this distinction, unlike the case of text, ways of connecting ideas could be proposed. 

For example, the most successful algorithm - it has been applied with great success in social and 
biological networks to detect communities or, more generally speaking, for the detection of structures which 
are distinguished on the basis of the different density of links embedded within the structure, compared with 
links commencing from the structure and ending in nodes of the remaining network - is the algorithm 
suggested by Newman & Girvan (2004), which is based on the concept of intermediate minimum paths 
(shortest-path betweenness). 

In summary, the steps of the algorithm are: 

1. Count the shortest distances (paths) between all pairs of vertices. 

2. Calculate the number of shortest paths that pass through each edge. 

3. Find the edge with the greatest number of paths and delete them from the network. 

4. After deleting repeat step 1. 

3.2 Assessing the Clarity of Modules 

As mentioned above, experts differ from beginners in terms of how they organize their knowledge. This 
means that the structures the expert develops ought to be more distinct, i.e. characterized by greater clarity 
and precision. Pellegrino, Chudowsky & Glaser (2001, pp. 77) present the maps portrayed in Figure 1 as a 
typical example of concept maps that were created by an expert and a novice respectively. By examining the 
maps on the level of semantics, it is obvious that the best treatment for the subject of motion comes from the 
expert. However, it is worth examining further, the clarity with which the expert forms structures compared 
to the beginner. 

The detection of modules in the context of a map should be accompanied by a valuation method 
measuring the clarity of these structures. For example, a widely acceptable approach in the area of network 
theory for quantitatively determining the clarity of structures is with the values of the modularity index. 
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Figure 1. Novices’ and experts’ concept maps 


The mathematical expression of this definition is: 




kk . 
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where m is the number of edges of the map, A (/ the adjacency matrix in the rows and columns of which is 0 
or 1 depending on whether there is an edge between vertices i and j, k and k, is the degree of vertices i and j 
(the number of edges that start from i and j), c, and c, the modules to which the vertices i and j belong 
respectively and 5(c„ cj) Kronecker Delta (for c, = c, equals 1, otherwise 0). 

The modularity indicator is equivalently expressed as: 

i 


where the first term in parentheses expresses the total number of edges between vertices of the same module 
(i module) and similarly, the second term expresses the total number of connections for the same module, if 
the links were randomly distributed. 

Let’s suppose that an algorithm (like the one in the previous section) has discovered two modules. We 
refer to the modules that are separated by the bold line, as indicated in the figure 1 . 
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Applying the index in order to evaluate the clarity of structure of the concept map of the novice in 
comparison to the expert concept map, we will take the following results: 

The concept map of the novice has 16 concepts and 15 edges (30 endings of edges) and there are two 
modules having 10 and 6 concepts. The first module has 9 edges (18 endings of edges) between its vertices, 
the second module has 5 edges (10 endings of edges) and there is 1 edge between the first and the second 
module. 

The clarity of modules is expressed as: 

Q = ^(e ii -a;) = 18/30 - (19/30) 2 + 10/30 - (11/30) 2 = 0,397 

i 

The clarity of modules, in the second case, is expressed as: 

Q = ^(e. . -a]) = 16/34 - (17/34) 2 + 16/34 - (17/34) 2 = 0,441 

i 

Applying the above method, we can prove in quantitative terms that the expert has developed more 
distinct structures in relation to the novice. The knowledge of expert is well-structured in relation to the 
knowledge of the novice. 


4. CONCLUSION 

Recent discoveries in the field of brain science and the latest constructions of mathematical concepts and 
tools in the area of network theory constitute fertile ground for the development and evaluation of learning 
activities as that of concept mapping. The classical methods of assessment can be enriched and greatly 
upgraded in light of discoveries about the operations of the human brain. Learner support in open type 
learning activities and the development of weak methods to solve problems can be done with the further 
refinement and development of a modular-based framework. 
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